Design verification (DV) of complex digital devices has developed rapidly as a technical discipline to keep pace with the immense progress in design methodology. As such, design verification addresses various facets of the efforts required stage through the register transfer logic model RTL stage and to the gate level implementation.
Some of these facets of the design verification task are listed below:
1. Functional verification of the implementation against the specification which verifies both the architectural and implementation specific device requirements;
2. Performance verification which evaluates the cycle count involved in multiple clock sequences against optimal performance;
3. Test plan design and test specification design: creation of stimulus/test cases;
4. Formal model checking and state machine analysis;
5. Creation of test coverage models (monitors); and
6. Bug tracking and reporting.
The design verification task is the effort of many contributors working jointly with the design team. Both the verification team and the design team are strongly committed to generating a design meeting all aspects of the device specification and establishing confidence that the new device is totally free of any failures (bugs) to operate exactly as specified.
The design verification effort for a complex integrated circuit may be addressed by at least tens if not over a hundred engineers and requires a high degree of coordination and mutual support if it is to be accomplished efficiently. A most crucial problem is to track tightly bugs that are identified, whether corrective action has occurred, and whether new bugs found have comprehended all the latest designs changes addressing previously found bugs. The verification effort suffers greatly when engineering effort is devoted to determining whether the last set of bugs identified have already been fixed. This type of invalid information is possible because the design is constantly being updated and simulations must proceed even though there is not complete awareness on the part of all involved which bugs have been fixed and which are still being addressed.
FIG. 1 illustrates the basic flow relating design models, simulation control, test patterns, test cases and simulation results according to the prior art. First, it will be helpful to clearly define the important types of models involved.
The Reference Functional Model (RFM) 100 is not a cycle-accurate model. As such the results from reference functional model simulations cannot be related on a clock-by-clock basis but only as the final result of the execution of an instruction. The Reference Functional Model 100 is simply an architectural model which exactly executes each instruction but without any intermediate information available during the execution. The Reference Functional Model 100 takes one instruction, executes it, provides the results and then moves to the next instruction. This Reference Functional Model 100 is mainly used to generate instruction-set combinatorial test cases.
The Golden C-Model (GCM) included in block 106 is a clock-cycle-accurate instruction set execution model that provides detailed information during the execution of each instruction and for each clock cycle of the execution, as each instruction is a multi-cycle path from its decode to its execution.
FIG. 1 illustrates the relationship between models and verification tasks. The Reference Functional Model 100 is used to drive directed random test case generation 101. Control files block 102 is developed by an engineer to direct the flow of the random test generation. The output of the random test generation 101 is a large number of test cases 103, typically thousands. These test cases exercise the device under test in an extensive manner that could not be addressed by manual generation and the result is that adherence to many functional requirements is quickly explored. The RTL simulation 104 and the Reference Functional Model/Golden C-Model 106 use these extensive test cases 103 to produce the simulation result noted by change detector 105. If the many thousands of random test cases 103 are effective and score hits on prevailing bugs, many thousands of failures 107 are generated. This kind of shot gun approach to revealing and identifying bugs in the design is indispensable in progressing in a timely fashion toward a bug-free design.
FIG. 2 illustrates the concepts of functional coverage and monitoring features and their interaction with test generation and simulation flow on a complex digital processor product according to the prior art. The central block element test generation and simulation 200 includes all the complex processes involved in the random as well as manual test generation and simulation using ever improving simulation software packages. The test generation and simulation 200 draws on two sources of input information: control files 201; and a pool of functional monitors 203. The remaining blocks emphasize the various aspects of functional coverage and bug monitoring. The test generation and simulation block 200 produces two main outputs: a passed test record 202; and a filed test record 204. Functional coverage report 205 generates a report on the functional coverage of the tests using the passed test record 202. Failed tests from failed test record 204 are processed in the test case filter block 206 and with an input from the monitor algorithms block 207. The test to debug block 208 generates details of the task of tests for debug. Engineering interaction with this flow proceeds from the test to debug block 208 and includes items in block 210, such as evaluation of the results of the most recent simulation and action items to be undertaken in the bug fixing process. Other engineering interaction includes evaluation of functional coverage results from functional coverage report block 205 and modification of control file 201.
The Reference Functional Model and the Golden C-Model of block 106 of FIG. 1 can both be linked with a post-processing environment (PPE) software for sorting the simulation results and generating bug tracking system reports. This linking depends upon the type of coverage or the bug. FIG. 3 illustrates the modification of the simulation and debug flow including a post-processing environment 319 employed in prior art to glean information from a simulation run. The prior art also employs the software monitor wrapper (SMW) 320 enabling more efficient use of simulation data.
Each simulation uses one or more test cases 300 and compares trace results from the individual RTL model 301 and Golden C-Model 302 simulations. The Golden C-Model simulation package 302 now typically includes special software probes within the software monitor wrapper (SMW) interface 320 sometimes called hooks that facilitate an interface designated software monitor wrapper (SMW) 320. The software monitor wrapper 320 processes the simulation results, presenting them in the form of software probe data for analysis by software external to the software monitor wrapper 320. In FIG. 3, two paths illustrate this feature. First, functional monitors 309 probe the functional results gathered at the software monitor wrapper interface 315, and pass these to the functional coverage database 313. Specification input block 318 illustrates manual interaction by engineers updating the functional monitors to account for a number of items. Among these are:
1. Specific functionality that will not be implemented in the present design and that will be designated unallowed states to the users; and
2. Functionality that will be implemented in a future revision of the design but will not be a part of the introductory device specifications.
Secondly, the Golden C-Model simulation 302 within the software monitor wrapper 320 generates input to simulation trace comparisons 304. Simulation trace comparisons 304 includes a change detector output that clearly marks differences between the results of RTL simulation 301 and Golden C-Model simulation 302. The detailed comparisons data report 305 includes a full record of pass and failed tests and the detailed output of the change detector function of simulation trace comparisons 304. The engineer makes use of these results processed in a pass/fail report 311 to arrive at a failed tests disposition 307. The engineer also uses these results in failure analysis 314. Passed tests 306 are accounted for in the functional coverage database 313.
Results of the failure analysis 314 are recorded in the bug tracking system (BTS) report 316. Bugs seen as fixed in bug tracking system report 316 are noted by issue fix confirmed block 310 and fixed bugs are discarded at 321.